Papers with rollout process
MGPO: Thinking with Images via Multi-Turn Grounding-Based Reinforcement Learning (2026.findings-acl)
Copied to clipboard
| Challenge: | State-of-the-art large multimodal models face challenges when processing high-resolution images, as these inputs are converted into enormous visual tokens, many of which are irrelevant to the downstream task. |
| Approach: | They propose a multi-turn grounding-based policy optimization framework that enables LMMs to iteratively focus on key visual regions by automatically cropping sub-images based on model-predicted grounding coordinates within a multiple-turn conversation framework. |
| Outcome: | The proposed framework improves on Qwen2.5-VL-7B with 21K samples and surpasses OpenAI’s o1 and GPT-4o models on the out-of-distribution (OOD) V* Bench. |
Chunks as Arms: Multi-Armed Bandit-Guided Sampling for Long-Context LLM Preference Optimization (2026.acl-long)
Copied to clipboard
Shaohua Duan, Pengcheng Huang, Xinze Li, Zhenghao Liu, Xiaoyuan Yi, Yukun Yan, Shuo Wang, Yu Gu, Ge Yu, Maosong Sun
| Challenge: | Recent studies have explored fine-tuning Large Language Models with synthetic data to enhance their long-context capabilities. |
| Approach: | They propose a framework that leverages a Multi-Armed Bandit rollout strategy to identify the most informative chunks from the given long context for sampling high-quality and diverse responses. |
| Outcome: | The proposed framework achieves 4% improvement on long-context reasoning benchmarks on Llama and Qwen. |